On General Adaptive Sparse Principal Component Analysis

The method of sparse principal component analysis (S-PCA) proposed by Zou, Hastie, and Tibshirani (2006) is an attractive approach to obtain sparse loadings in principal component analysis (PCA). S-PCA was motivated by reformulating PCA as a least-squares problem so that a lasso penalty on the loading coefficients can be applied. In this article, we propose new estimates to improve S-PCA in the following two aspects. First, we propose a method of simple adaptive sparse principal component analysis (SAS-PCA), which uses the adaptive lasso penalty (Zou 2006; Wang, Li, and Jiang 2007) instead of the lasso penalty in S-PCA. Second, we replace the least-squares objective function in S-PCA by a general least-squares objective function. This formulation allows us to study many related sparse PCA estimators under a unified theoretical framework and leads to the method of general adaptive sparse principal component analysis (GAS-PCA). Compared with SAS-PCA, GAS-PCA enjoys much improved finite sample performance. In addition, we show that, when a BIC-type criterion is used for selecting the tuning parameters, the resulting estimates are consistent in variable selection. Numerical studies are conducted to compare the finite sample performance of various competing methods. Datasets and computer code are available in the online supplements.